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Abstract- Intelligent robot the concepts arrives from putting 
some idea into robots mind. Our robot intelligence is for 
recognizing different patterns assigned to different class of 
objects. This robot is simply a pick and place robot but having 
intelligence in it. The intelligence is added using image 
processing techniques. Image processing technique provides a 
reliable processing for the robot. The signal generated from 
processing the real time images is used as input to end -effector 
for doing the assigned work. These real time images are taken 
by a web-cam mounted on robotic arm. This added intelligence 
will provide a better regulated pick and place robot which can 
be used for different types of pick and place operations. 
Pattern recognizing is easier to implement than a object 
recognizing techniques and hence it will be suitable for small 
scale industries and for undeveloped area to start with robotic 
automation. 

Index Term- Image Processing Based Robot, Intelligent 
robot, pick and place robot, SIFT algorithm 


I. INTRODUCTION 

Intelligent robot as the name suggest, a robot with some 
added intelligent. A pick and place robot provided with some 
intelligence using image processing technique. Intelligence 
is mainly concern with recognizing some predefine pattern to 
obtain the class of the object. Based on the class of the object 
and placing it to predefined place. Since pattern recognizing 
is easier to implement the object recognizing technique, it 
will be helpful small scale industries and industries in 
undeveloped areas to get help of automation to increase the 
production in less investment. Image will be obtained in real 
time to match with pre-defined pattern. Once the match is 
found the object will be picked by the robotic end effectors to 
place it on its pre-defined place. We can assign the pattern to 
the object based on their type, class or requirements. Due to 
this we can increase the productivity of pick and place robot 
which can now be used to pick and place the object of 
different kind at different places. Because normally pick and 
place robots are used to pick the object from a specified place 
and place it to some different place any type of object coming 
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will picked and placed on same place. So by this technique 
we can increase the productivity. 


II. PATTERN DETECTION 

A. Pattern Detection 

In this paper some pattern will be detected to detect the type 
of object or class of object. This pattern detection can be done 
by two methods one is feature based and other is template 
based approach. We can use feature base approach since the 
search image might get transformed in some manner. In this 
approach the entire template is not considered and features 
are extracted from the image for matching purpose. This 
extracted points will matched in real time to find the required 
object. Once the extracted feature come in area of action the 
pattern will be detected and further action can be taken. We 
can use OpenCV library to implement the pattern detection 
algorithm. 



Fig.l 


B. Feature Extracting Algorithm 

We can use SIFT(Scale-In variant Feature Transform) 
algorithm for detecting the feature in the image. We will 
provide the image of the pre-defined patterns at initial phase. 
SIFT algorithm extracts the feature of image in terms of 
keypoints. A pattern with contrast, and gradient variation in 
color to get more no. of keypoints. Because more gradient 
variation will produce higher number of keypoints resulting 
better detection image. As per our study the background 
should be a single color lighter than foreground color. This 
produces good keypoints for matching. White background is 
suggested for good results. Now these points are matched by 
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calculating the distance from the neighboring points. Based 
on the match it gives the no. of match points found in 
process. We can set a threshold value after which it can be 
considered as good match. This will avoid the false 
triggering. As per our study the working are should not be 
directly exposed to light i.e. a direct bright light should be 
avoided in the area for better matching. Light should fall 
directly but it should be enough for the camera. We should 
also give as many templates as possible to get the larger 
feature points. SIFT algorithm is scale invariant hence it 
provide a good result than some other algorithm in case of 
scale variation. 
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Fig. 2 


C. Image Matching 

Now we have to continuously provide the image as input to 
match and get the output. This can be done by providing a 
continuous video feed from a high resolution camera. Now 
the frame of the video feed can be used by processing it frame 
by frame. This means we will take a frame from the video 
process it if match found then action will be taken otherwise 
next frame will be processed. We can use FLANN technique 
for this because we will be comparing a set of images. Our 
study suggests that the camera should be at least 30fps to 
produces a good result. OpenCV provides the technique to 
process the frame without saving it, this saves a lot of 
memory space and gives a fast processing. 

D. SIFT Algorithm 

SIFT algorithm uses feature point detection for matching two 
images. Feature points are which provides needful 
information from the image which can be used get the task 
done. Some points which do not vary even on transforming 
the scale are called as keypoints or interest point. 
Keypoints are the information extracted from different 
locations of an image. These points consist of corners, 
junction and the region which have the sharp change from 
surrounding i.e. blob points. The keypoints have a large set 
of information about the image. These information doesn’t 
change if we transform the image and hence this is Scale 
Invariant Feature Transform. The keypoints are also 
invariant to addition of noise, illumination and viewing 


angle. Hence this keypoints are stable key point which can be 
used to do various further processes. 

Keypoints information can be variation in intensity or 
difference between two adjacent pixel etc. According to C. 
Schmidt [1J keypoint detectors can be of different type based 
on change in contour, change in intensity and change in 
parametric factors . 

Contour based detectors detect the point on some 2D curve 
like change in shape, corners, junction of two curves etc. In 
intensity based detectors the computation of the interest point 
is done from the gray level image. And Parametric method 
based on the parametric function of an image. 

SIFT algorithm is mainly based on an operator proposed by 
Lowe L2J known as SIFT operator. This operator being scale 
invariant also performs large number of feature extraction 
and identification. 

Keypoint descriptors are used for comparing the keypoints of 
two images. The keypoint descriptors are the information 
collected from neighborhood of the image. This information 
can be used to distinguish between two keypoints. For 
generating keypoint descriptor normalized Gaussian filter is 
used for smoothening process. After this process the 
neighborhood is divided small regions. These regions are 
called as orientation histogram and these have the size of 
4*4. By the above process keypoint descriptors are produced 
which are used for keypoint detection. 

So, greater the number of key points more accurate the result 
will be. Hence by providing a large database of an image we 
extract more number of key points. Images of the same object 
taken from different perspective give larger number and 
different type of keypoints. The initial object detection result 
may not be accurate but as the number of detected keypoint 
increases with process the result becomes more accurate. 

Due to all this regions SIFT algorithm becomes a useful 
algorithm for object detection programming. Also this 
algorithm is time efficient and consumes less memory. Due 
to its scale invariant approach this is a handy algorithm for 
pattern detecting intelligent robot. We are OpenCV library to 
implement our pattern detection program. OpenCV being a 
dedicated computer vision library provides an efficient 
solution for implementing image processing programs . 


E. SIFT In OpenCV 

There are different functionality of SIFT which can be 
availed in OpenCV . These functionalities are used to process 
an image in OpenCV using SIFT. 

Following are some functions which we are using in our 
program. 

sift.detect() this function is used to find keypoints in an 
image. To only detect some specific part of the image we can 
use mask for that. 
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sift.computeO this function is used to get the descriptors 
from the image. This function is mainly used after 
sift.de tect() function. 

sift.detectAndcompute() this function detects the keypoints 
as well as computes for descriptors in one function only. 

cv2.drawKeyPoints() this function in OpenCV is used to 
draw small circle for keypoints. 

cv2.DRAW_MATCHES_FLAGS_DRAW_RICH_KEY 

POINTS0 this function is used to draw keypoint size circles 
and show the orientation. 

III. IMPLEMENTATION 

A. Block Diagram 


IV. CONCLUSION 

Using SIFT image detection algorithm we were able to make 
a autonomous robot to perform pick and place task. 

It gives very accurate result. 

V. FUTURE SCOPE 

This pattern detector robot can be later use as a cooking 
robot. Since vessels and method for cooking are repetitive, 
this robot can be used for the cooking purpose by increasing 
some intelligent. 
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Fig. 3. Block Diagram 

Fig. 3 above shows the block diagram of our image 
processing based intelligent robot. Here webcam will operate 
as eye of the robot which capture image provide it to the 
processing unit i.e. a laptop or a desktop system. This system 
will process the image and accordingly pass the signal to 
microcontroller which will make the arm to do the 
predefined task. 

B. Working 

Here the real time image from webcam will be given to 
processing unit for process. The keypoints from the image 
will be extracted and then it will be computed for keypoints 
descriptors. After this part these keypoint descriptors will be 
compared with the keypoint descriptor of template image. If 
the threshold amount of keypoint match is found then it will 
be considered as positive result i.e. image detected. Once the 
image is detected this signal will be passed to 
microcontroller. Now work of microcontroller is to make the 
robotic arm pick the object from its location and place it to 
the desired location. This task will be performed by 
microcontroller by controlling the different 
servomotor of the robotic arm. Servo motor will provide high 
accuracy to performing the task. In this way automated and 
selective pick and place robot can be implanted using image 
processing. 
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